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ABSTRACT 


Background: Artificial intelligence literacy (AIL) tools have been developed to evaluate whether 
experts or non-experts can understand and apply AI in their respective professions. However, no 
model or study exists for assessing AIL in pediatric dentists. Aim: This study aimed to assess the 
literacy of pediatric dentists in artificial intelligence based on a phrase recognition, knowledge, and 
comprehension model through a mixed methods approach. Methods: A list of 13 Al-related phrases 
listed through expert-consensus was sent to participants who rated their interest in AI as three or 
higher on a 5-point Likert scale. The participants were asked to recognize each phrase. If they 
answered yes, they were further evaluated for knowledge and comprehension for that phrase 
qualitatively. If they did not recognize the phrase or incorrectly described it (knowledge), then the 
literacy assessment for that phrase was not taken to the next step (knowledge or comprehension). 
Data for 31 participants were tabulated and sent for further analysis. Results: 12 out of 31 
participants recognized, had knowledge, and comprehended the 13-AI related phrases correctly. Chi- 
square tests did not reveal a significant association between the three domains. Spearman's 
correlation coefficient showed high internal consistency between the knowledge and comprehension 
responses of the participants. Conclusion: AIL of pediatric dentists can be evaluated based on 
phrase recognition, knowledge, and comprehension model which can be refined by its application on 
a larger sample. 
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INTRODUCTION 


Artificial intelligence literacy (AIL) is defined as ‘a set of competencies that enables individuals to 
critically evaluate AI technologies, communicate and collaborate effectively with AI, and use AI as a 
tool online, at home, and in the workplace’.' The speciality domain of AIL has garnered interest in 
recent times as it is gradually becoming imperative for machine learning technology to take 
significant control of daily living and workplace. Thus, it becomes a growing need to ensure that 
non-experts understand the basics of AI conceptually and can have their own perspective and practice 
of technological advances. 


Research involving artificial intelligence in pediatric dentistry has been carried out since the 1990s.” 
However, it becomes essential to evaluate if the terminologies related to the field of AI that are 
mentioned in these research papers are understood by interested readers. The objective of this 
preliminary investigative paper was two-fold: a) to evaluate the AIL of pediatric dentists, and b) to 
give a framework to a literacy assessment scale based on recognition, knowledge, and 
comprehension of Al-related phrases. 


METHODOLOGY 


To accomplish our objectives, a mixed methods approach was used (Figure 1). The reporting of this 
study is based on the recommendations of the ASSESS tool.* Review articles were searched across 
relevant databases for articles published in English that focused on AI in pediatric dentistry. Two 
systematic reviews and two narrative reviews published since 2023 were chosen for full-text 
screening.>® 


Three authors independently listed down all phrases related to AI that they felt pediatric dentists must 
be aware about. These authors have experience in AI-based applications and machine learning tools 
for an average of 6.90 years. All three lists were then compared. A total of 24 phrases were listed. If a 
phrase was mentioned in at least two of the three lists, it was included as a test element. If a phrase 
was included only in one list, that its inclusion was decided by consensus. Based on this criteria, the 
final set of phrases that were to be tested for AI literacy in pediatric dentists came to a count of 13. (A 
list of excluded terms can be available upon a request to the authors). 
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SEARCH THE LATEST SYSTEMATIC QUESTIONNAIRE STATING INTEREST IN Al (RATED 
REVIEWS ON THE USE OF Al Ww A ON A %POINT LUKERT SCALE) WORK 
FIELD OF UNTEREST EXPERIENCE, MODE OF COMMUNICATION 


LISTING DOWN BASIC & CONCEPTUAL 
PHRASES RELATED TO Al BY AT LEAST 
THREE EXPERTS INDEPENDENTLY 
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PHRASE IS NOT ASSESSED FURTHER 


COMPREHENSION 


Figure 1 Mixed Methods Approach 


A customized digital form was sent to a sample population of pediatric dentists. This included three 
questions. The first was based on a Likert scale where they were asked to rate their interest in AI in 
the field of pediatric dentists. In the second question, they were asked to mention their years of 
experience. The final question was attached with a consent to participate further in the study. 
Participants were asked to mention their preferred mode of communication. A total of 42 such 
questionnaires were sent through snowball sampling technique. Participants who scored less than 
three in the first question were excluded from the study. Participants who scored three or more were 
contacted based on their preferred mode of communication. 


After reaching out the participants in the second phase of our research, the participants were asked 
about the 13 AI phrases on three fronts: recognition, knowledge, and comprehension. If a participant 
could recognize the word (answered as yes or no), then they would be asked to describe or define it 
(knowledge). If this answer was correct, they were then asked to give a relevant dental example 
about the application of that phrase (comprehension). If the answer in the recognition and knowledge 
sections were incorrect, then the interview for that phrase would not proceed to the subsequent 
stages. 


After the evaluation of all the participants for the 13 phrases, the data was quantitatively evaluated 
for recognition and qualitatively for knowledge and comprehension. The answers of knowledge and 
comprehension were given thematic coding, and common terminologies used by the participants 
were used to identify the reason behind an answer being correct or incorrect. If a participant could 
not answer, then it was coded by the statement ‘could not explain.’ The tabulated data was then sent 
for statistical analysis. 
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RESULTS 


Out of the 42 questionnaires sent, 33 scored three or more when asked about their interest in AI in 
pediatric dentistry. Two of the participants could not be reached or contacted. Thus, the response rate 
for our questionnaire was 73.80% and statistical analysis was thus done for 31 responses (Table 1). 
The mean experience of participants in the field of pediatric dentistry was 8.48 years. 


Descriptive statistics show that pediatric dentists could recognize an average of 20 phrases (mean: 
19.69). However, when asked to describe or define the recognized terms, 13 participants (mean: 
13.15) could achieve the task satisfactorily. 12 participants (mean: 12.08) out of the 13 who 
described the meaning of the phrase could also give a correct example of its application in dentistry 
or pediatric dentistry (Table 2). 


Chi-square statistics were carried out to analyze the relation between recognition, knowledge, and 
comprehension of the AI phrases. None of the associations showed statistical significance. Spearman 
correlation coefficient was calculated for each of the three group to compare the direction and 
strength of the associations. Moderate positive correlation was show between recognition and 
knowledge, and recognition and comprehension. A strong correlation was shown between knowledge 
and comprehension indicating that those who had correct knowledge of a phrase were also likely to 
demonstrate a better comprehension of the same (Table 3). 


Since ‘knowledge’ and ‘comprehension’ are two homogenous constructs that contribute to a common 
underlying factor (assessing AI literacy), it was important to calculate the reliability between the two 
variables. A Cronbach’s Alpha score of 0.993 indicated a high internal consistency between 
knowledge and comprehension. 
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PHRASE Recogn | KNOWLEDGE COMPREHENSION 
ize (y/n) | Correct Incorrect Correct Incorrect 
Al Y: 31 Robots (n=3) Technologi | Patient management software (n=1) - 
N: 0 Machine cal advance | Behavior Guidance (n=8) 
intelligence/assistance (n= 7) Caries diagnosis (13) 
(n= 21) Radiographic advances (n=2) 
Narrow AI Y: 3 Weak AI (n=2) - Caries diagnosis (n=3) - 
N: 28 Task specific (n=1) 
Machine Y: 31 Learning based on input | Same as AI | Cephalometric analysis (n=2) - 
Learning N: 0 data (n= 4) (n=21) Caries diagnosis (n=4) 
Algorithm (n= 4) Teeth identification (n=4) 
Coding (n=2) 
Deep Learning | Y: 26 Advanced form of Same as AI | - Could not 
N:5 machine learning (n=3) | (n=23) explain 
(n=3) 
Big Data Y: 2 Large dataset (n=2) - Multiple radiographic images (n=1) - 
N: 29 Multiple clinical images of caries 
(n=1) 
Algorithm Y: 31 Flowchart (n=11) Numerical | Treatment options (n=16) - 
N: 0 Planned process in value (n=2) 
computers (n=5) Computer 
code (n=9) 
Software 
language 
(n=4) 
Virtual Reality | Y: 31 VR Box/glasses (n=30) | - Behavior Guidance (n=31) - 
N: 0 Being in a fictional 
world (n=1) 
Augmented Y: 27 Digital graphics overlap | Same as Interactive dental games (n=6) Hologram 
Reality N:4 with real world (n=10) | VR (n=17) | Dental education (n=2) (n=1) 
Haptic simulation (n=1) 
Remote Y: 11 Digitally monitoring - Tele-dentistry (n=7) Could not 
Monitoring N: 20 patient’s treatment Caries risk assessment (n=1) explain 
outcomes (n=8) (n=3) 
Intraoral scans (n=3) 
Neural Y: 6 Mapping of data (n=3) | Neuronal Patterns learned by AI from multiple | Could not 
Network N: 25 Digital mimicry of connection | images (n=3) explain 
brain (n=1) s (n=2) Risk factors for caries (n=1) (n=2) 
Large Y: 23 ChatGPT (n=23) - Manuscript writing (n=6) For patient 
Language N: 8 Statistical analysis (n=12) education 
Model Digital content generation (n=1) (n=4) 
In Silico Y: 14 Studies using - Finite element analysis studies (n=5) | Could not 
N: 17 computers (n=8) Using a software for prognosis of explain 
Simulating clinical treatment (n=2) (n=3) 
environment digitally Diagnostic accuracy of an AI model 
(n=6) (n=4) 
Gamification Y: 20 Child is allowed to play | - Behavior guidance (n=20) - 
N: 11 games during dental 
treatment (n=20) 
Table 1 Qualitative Analysis 
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PHRASE RECOGNIZED KNOWLEDGE COMPREHENSION 
Yes No Correct Incorrect Correct Incorrect 
Artificial 31 00 24 07 24 00 
Intelligence 
Narrow AI 03 28 03 00 03 00 
Machine 31 00 10 21 10 00 
Learning 
Deep Learning 26 05 03 23 00 03 
Big Data 02 29 02 00 02 00 
Algorithm 31 00 16 15 16 00 
Virtual Reality 31 00 31 00 31 00 
Augmented 27 04 10 17 09 01 
Reality 
Remote 11 20 11 00 08 03 
Monitoring 
Neural Network | 06 25 04 02 04 02 
Large Language | 23 08 23 00 19 04 
Model 
In Silico 14 17 14 00 ll 03 
Gamification 20 ll 20 00 20 00 
Table 2 Qualitative Analysis 
Variables Chi-square p-value Spearman p-value 
Statistic Correlation 
Recognition vs Knowledge 95.87 0.316 0.613 0.026 
Recognition vs Comprehension 117 0.261 0.627 0.022 
Knowledge vs Comprehension 130 0.251 0.967 <0.001 


Table 3 Tests of significance and association 
DISCUSSION 


A total of 16 assessment tools for AIL have been recently analyzed in a systematic review. However, 
none of them have been specifically developed for the field of dentistry. Though the Scale for 
Assessment of Non-Experts’ AI Literacy (SNAIL) exists, its content validity remains questionable 
with low reliability.” 


Word recognition and comprehension has been previously used for assessing literacy in healthcare.'° 
Assessment of literacy in a subject have been evaluated based on conventional models and 
frameworks like Bloom’s taxonomy, The Five Big Ideas, Digital Literacy Frameworks, Socio- 
technical Perspective, Ecological Systems Theory, Computational Thinking, TPACK Framework, 
etc.'!' The common constructs in these frameworks are “use and apply” (82%), “know and 
understand” (73%), recognize and create (64%).!? The model used in our preliminary investigation 
included three of these factors: recognition, knowledge, and comprehension. Though this model can 


25 
Dave J., Garg V., Alwani A. Artificial intelligence literacy among pediatric dentists based on phrase 


recognition, knowledge, and comprehension: A preliminary investigation. The Quadrant. 
2024;2(3):20-27.https://doi.org/10.528 1/zenodo.13709765 


THE QUADRANT — Volume 2 Issue 3 — May-Aug 2024 


be applied to assess Al-relation phrase proficiency in any field, we used pediatric dentists as our 
sample as it was the field of expertise of the evaluators in this study. 


The results of our study showed that only 12 of the 31 participants recognized, had knowledge, and 
could comprehend the 13 Al-related phrases correctly. The cross-sectional design of our study can 
help in understanding the current status of AIL in a given field. Though the scale comprehensively 
illustrates both qualitative and quantitative assessment, it lacks cross-cultural validity, measurement 
invariance, and measurement error which is a limitation worthy of mention. These can be overcome 
by assessment of AIL on a larger sample size. 
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